Standardizing quality assessment of observational studies for decision making in health care.

This report summarizes a roundtable discussion of managed care experts and academic pharmacoeconomists, held in March 2008 with the objective of defining action steps to overcome barriers to the incorporation of real-world data into health care decision making. The roundtable meeting was the result of an initial program in July 2007 where managed care decision makers and pharmacoeconomic experts defined these barriers.1 Real-world data in this context were characterized as data not routinely collected in Phase III drug registration studies, including administrative claims data, patient registries, large simple trials, resource use collection alongside clinical trials, and electronic medical records.1,2 These data are considered along with standard safety, efficacy, and pricing information. However, one conclusion from the first roundtable discussion was that concerns around the quality assessment of such data have hindered widespread use in decision making. The Foundation of Managed Care Pharmacy (FMCP) recognized the value of real-world data in its AMCP Format for Formulary Submissions, version 2.1, a structured outline for the presentation of information by pharmaceutical companies on their products to managed care decision makers.3 As a sponsor and developer of this standard, FMCP has invested in the adoption of the AMCP Format by manufacturers and health plans. Following a broad communication strategy, over 50 training seminars to managed care pharmacists on using the AMCP Format have been held.4 The acceptance of the AMCP Format approach has improved since its inception in the year 2000; however, its impact on formulary decisions is still in its infancy. A recent survey found that approximately one-third of all pharmacy directors request information from drug manufacturers in a form that is consistent with the AMCP Format.5 While information delivered in the dossier concerning the safety and efficacy for labeled use was perceived by the health plans to be mostly satisfactory, the information related to off-label use, costs, and benefits was perceived as incomplete, lacking in clarity, and potentially biased. These data indicate that despite extensive efforts, the implementation process has not yet been fully effective in promoting the utilization of real-world data. The struggle for adoption of new processes in health care delivery is not new. In the United Kingdom (UK), an initiative called PARiHS (Promoting Action on Research Implementation in Health Services) has addressed the importance of process implementation, and in the United States a framework called REP (Replicating Effective Programs) has been established.6,7 Both present conceptual frameworks and their application for transfer of health care knowledge from research into practice or from one organization to another from the perspective of implementation sciences. Within the circles of implementation science, a theory of “sticky knowledge” has been propagated, referring to the inherent resistance of the old process against the new. This leads to inefficient knowledge transfer as a significant barrier to implementation of new processes into the health care environment.6-9 These publications highlight that a structured process must support the effective introduction of an innovation. This process consists of actions before the actual change (learning before doing) and actions after the first day of usage (learning by doing). The process involves the source of the knowledge, the recipient of the knowledge and the environment for the change, and it should be guided by personal facilitation. As related to the incorporation of real-world data into formulary decision making, the creation of a process or technique for using real-world data is only a starting point. Implementation requires a multistep process, to be carefully planned and executed. Thus, while the 2007 meeting of managed care experts and academic pharmacoeconomists recognized the importance of real-world data, identified the potential barriers, and recommended methodological approaches to overcome those barriers, the issue of process implementation was not fully addressed. The methods and the planning of process implementation were targeted in 2008, where the participants focused on (a) integration of currently available tools for quality assessment of realworld data studies into 1 standard instrument; (b) creation of an implementation process for the dissemination of the instrument, including its validation by peer groups; and (c) establishment of a training certificate program to educate the potential users of such a tool. Here, we report the progress of the work and the resulting integrated action plan to support the developing body of research on this topic.